Python: Getting Started

Simple Calculations¶

In addition to all the more complex things you can do in Python, it is also useful for completing simple mathematical operations.

Addition and Subtraction

Add numbers together like this:

Notice that you don't need to include = in order to make the operation work.

Subtract numbers like this:

Division

Divide numbers like this:

Remember that Python will return an error if you try to divide by 0!

Modulo Division

Perform modulo division like this:

Multiplication

Multiply numbers like this:

Add exponents to numbers like this:

Order of Operations¶

Just like everywhere else, Python works on a specific order of operations. That order is:

Parentheses Exponents Multiplication Addition Subtraction

If you remember being taught PEMDAS, then this is the same thing! All the operations in Python work with levels of parentheses, so if you wanted to add two numbers together, and then add another number to the result, the code would look like this:

Things get more complicated when we add more terms to the equation, but the principle remains the same: if you open a set of parentheses, make sure you close it too. Here's an example that uses all the PEMDAS possibilities:

Data Types¶

There are lots of different types of data in the world, and Python groups that data into several categories.

Boolean (bool):

• Any data which can be expressed as either True or False.
• Used when comparing two values. For example, if you enter 10 > 9, Python will return True.

String (str):

• Data that involves text — either letters, numbers, or special characters.
• Strings are enclosed in either single- or double-quotation marks: "string-1" or 'string-2'.

Numeric (int, float, complex):

• Data that can be expressed numerically.
• An integer, or int, is a whole number, positive or negative, without decimals, of unlimited length: 123.
• A floating-point number, or float, is a number, positive or negative, containing one or more decimals: 123.01.
• A complex number, or complex, are imaginary numbers, designated by a j: (3 + 6j).

Sequence (list, tuple, set):

• Data that is a collection of discrete items.
• A list is collection that is ordered and changeable. It's designated using square brackets [], and items can be of different data types: ["red", 1, 1.03, 1].
• A tuple is a collection which is ordered and unchangeable. It's designated using parentheses (): ("red", 1, 1.03, 1).
• A set is a collection which is unordered, unchangeable, and does not permit duplicate items. It's designated using curly brackets {}: {"red", 1, 1.03}.

Mapping (dict):

• Dictionaries store data in key-value pairs. They're designated using curly brackets {}, like a set, but notice that keys and values are associated with each other using a colon :. Each pair is separated from the next using a comma ,.

dict1 = {
    "department": "quindio", 
    "property_type": "house", 
    "price_usd": 330899.98
}

Binary (bytes, bytearray, memoryview):

• Used to manipulate and display binary data. That is, data that can be expressed with integers represented with base 2.
• Unlike the other data types described above, binary types are not human-readable.

Lists¶

In Python, a list is a collection of data that stores multiple items in a single variable. These items must be ordered, able to be changed, and can be duplicated. A list can store data of multiple types; not all the items in the list need to be the same type.

Creating Lists¶

Lists can be as long or as short as you like. Let's create a short list based on data from the Colombian real estate market to give us something to work with.

Lists are written with square brackets. Code for a short list that shows the price of houses in US dollars looks like this:

Working with Lists¶

After you've created a list, you can access any item on the list by referring to the item's index number. Keep in mind that in Python, the first item in a list is always 0.

Let's access the second item of our price_usd list.

Practice

Try it yourself! Create and print a list that shows the area of the houses, called area_m2. Include the items 187.0, 82.0, and 235.0.

If we want to access the an item at the end of the list, we can use negative indexing. In negative indexing, -1 refers to the last item, -2 to the second to last, and so on.

Let's access the last item in our department list.

Practice

Try accessing the second item in your area_m2 list.

Try accessing the last item in the same list.

Appending Items¶

It's also possible to add an item to a list that already exists using the append method like this:

Practice

Add the item 195.0 to your area_m2 list.

Aggregating Items¶

We can also aggregate items on a list to make analyzing the list more useful. For example, if we wanted to know the total value in US dollars of the houses on our price_usd list, we could use the sum method.

We might also be interested in the average value in US dollars of the houses on the same list. To find the average, we add the len method to the sum method.

Practice

Try it yourself! Calculate the total area of the houses on your area_m2 list, and find the average area of all the houses on the list.

Zipping Items¶

Finally, it might be useful to combine -- or zip -- two lists together. For example, we might want to create a new list that pairs the values in our price_usd list with our area_m2 list. To do that, we use the zip method. The code looks like this:

You might have noticed that the above code involving putting one list (in this case, new_list) inside another list (in this case, list). This approach is called a generator, and we'll come back to what that is and how it works later in the course.

You might have noticed that the above code involving putting one list (in this case, new_list) inside another list (in this case, list). This approach is called a generator, and we'll come back to what that is and how it works later in the course.

Practice

Try it yourself! Create a list called area_m2 that includes the terms 235.0, 130.0, and 137.0, then create another list called price_cop that includes the terms 400000000.0, 850000000.0, and 475000000.0. Then zip them together to create a new list called area_price, and print the result.

Python for Loops¶

A for Loop is used for executing a set of statements for each item in a list.

Working with for Loops¶

There can be as many statements as there are items in the list, but to keep things manageable, let's use our list of real estate values in Colombia.

We might want to see each of the values in the list, so we insert a for Loop:

Note that the print command is indented.

Practice

Try it yourself using the area_m2 list:

Python Dictionaries¶

In Python, a dictionary is a collection of data that occurs in an order, is able to be changed, and does not allow duplicates. Data in a dictionary are always presented as keys and values, and those key-value pairs cannot be duplicated in the dataset.

Creating Dictionaries¶

Dictionaries can be as big or as small as you like. Let's create a small dictionary based on data from the Colombian real estate market to give us something to work with.

Dictionaries are written with curly brackets, with key-value pairs inside. Code for a small dictionary looks like this:

Practice

Try it yourself! Create and print a dictionary called bogota with the key-value pairs "price_usd": 121,555.09, "area_m2": 82.0, and "property_type": "house"

Working with Dictionaries¶

After you've created a dictionary, you can access any item by using its key name inside square brackets.

Going back to our example dictionary, let's access the value for "department".

Practice

Try accessing the value for price_usd in the Bogotá dictionary you created above.

You can also use get to retrieve a value. That looks like this:

Practice

Now try accessing the value for area_m2 using the get method, and print the result.

Dictionary Keys¶

Sometimes you want to know what the keys are in the dictionary, or you want to iterate through the dictionary. In such cases, you need to write code to access all keys in a dictionary by utilizing the keys method:

If you need to use the keys in a list, you can transform the output:

You can also iterate though colomdict.keys() without converting it to a list:

Practice

print the value of each key in the colomdict dictionary

JSON¶

JSON stands for Java Script Object Notation, and it's a text format for storing and transporting data.

Working with JSON¶

JSON works by creating key-value pairs, where the key is data that can be represented by letters (called a string). JSON values can be strings, numbers, objects, arrays, boolean data, or null. JSON usually comes as a list of dictionaries, which look like this:

Here's an example from our colombia-real-estate-1 dataset with two key-value pairs that both include string values:

Here's an simplified example from our colombia-real-estate-1 dataset with two key-value pairs that both include string values:

JSON pairs with numbers look like this:

When you mix more than one type of value, it looks like this:

References & Further Reading¶

• A guide to basic math operations in Python
• Python documentation on built-in data types
• Summary of Python data types
• Tutorial on type conversion in Python
• A description of how dictionaries work in Python
• An introduction to JSON
• An introduction to lists in Python
• How to zip lists
• Calculating mean, median, and mode in Python
• A brief tutorial of For Loops

Copyright © 2022 WorldQuant University. This content is licensed solely for personal use. Redistribution or publication of this material is strictly prohibited.

Python: Advanced

Strings¶

What's a string? ¶

Recall that a string is any kind of information that can be represented with letters.

Working with strings ¶

When working with data, often files and directories have names that fit a pattern. For example, data on property prices in Colombia and Mexico might be stored in files named:

1. colombia-real-estate-1.csv
2. colombia-real-estate-2.csv
3. colombia-real-estate-3.csv
4. mexico-city-real-estate-1.csv
5. mexico-city-real-estate-2.csv
6. mexico-city-real-estate-3.csv
7. mexico-city-real-estate-4.csv
8. mexico-city-real-estate-5.csv
9. mexico-city-test-features.csv
10. mexico-city-test-labels.csv

When the list of files is short like this one, it's not difficult to find the ones we want, but if the list were longer, we might need some help. If we're only interested in finding files that deal with Mexico, we could search the files for files beginning with mexico-city-real-estate-. To do this, we'll use the .glob function. The code looks like this:

The .glob function allows for pattern matching. In this example [0-9] allows for any digit between 0 and 9, but there are lots of other patterns that .glob can find. Here are a few of the more common ones:

• * Match any number of characters
• ? Match a single character of any kind
• [a-z] Match any lower case alphabetical character in the current locale
• [A-Z] Match any upper case alphabetical character in the current locale
• [!a-z] Do not match any lower case alphabetical character in the current locale

So, if we wanted to find all the files from Mexico City, we would use code like this:

Practice

Try it yourself! Find only the data files containing the word test.

So far, you have only searched for files in one specific directory. It's also possible to search for files in subdirectories. To get a listing of all notebook files starting from the directory above this one and all others below it, you can use:

Manipulating Strings¶

We can split a string on a specific character, transforming it into a list:

This can be useful for a set of strings with similar naming conventions for us to access specific information. For example, we can collect the digit as the file number in the string:

Another useful function to manipulate a string is replace. We can use it to replace any element in the string with another item. Note we put what we what to replace in the front and what we replace it with in the back:

Practice

Change "mexico-city-real-estate-1" to "mexico-city-real-estate" using replace. Hint: To delete a character, we can replace it with an empty string ("").

Working with f-strings ¶

We usually use print to examine output in Python, but most of the examples we've been printing have been relatively short. Formatted strings are helpful for all sorts of reasons, but when we're assembling and formatting a long string, using the print function can be difficult and time-consuming. Along the same lines, it's also useful to directly evaluate variables and expressions within strings. To do those things, we create f"" strings. The code looks like this:

Practice

Mexico-Tenochtitlan was established on 13 March 1325; use f-strings to indicate how long ago that was.

Iterators and Iterables¶

A list is a container with a countable number of values. Because that's true, a list is an iterable, meaning that we can iterate through it one item at a time. In other words, iterators retrieve these values only when we ask for them. If we try to bring in a large database — over a million values, for example — asking for every action to be applied to every value will take up a huge amount of memory. Iterators are helpful because they allow us to free up memory to use for other tasks. We'll spend more time working with databases later on, but for now, let's take a look at some code:

Setting aside the first two lines of code, we have a method which has returned a list of four databases. If we want to examine each database by itself, we can create a variable called results, and then try to print it.

That doesn't seem like much of anything, but if we add the iterator next(), we'll get back something more useful.

That makes much more sense! As you can see, this returns the first row. If we do it again, we'll get the second row:

We can keep doing this until we get to the end of the list, at which point we'll get an error telling us that there's nothing left to print. Every time we use the next() method, we're using it as an iterator to iterate through our iterable.

List Comprehension ¶

List comprehension is used to iterate through lists without explicitly writing loops, which is especially useful for filtering data according to a specific condition.

Let's take a look at a list that shows property prices in Mexican pesos.

But maybe we're interested in comparing these prices to property values in Colombia. To do that, we'll need to figure out how to express the data on our list in Colombian pesos. We can use a for loop to make the conversion based on an exchange rate of 1 Mexican peso to 190 Colombian pesos. The code looks like this:

But what if we could do the same thing, but using fewer lines? That's what list comprehension is for. The code looks like this:

We can use list comprehension to find all the house entries in this list of properties, like this:

Practice

Explore the list records in the list, and find all entries located in Tlalpan

Functions¶

When we code in Python, we want to create readable programs. One of the easiest ways to make a program readable is by not repeating sections of code that do the same thing. We do that by using functions. For example, you might have surface area of a property in square meters, but you want to see it in square feet. Keeping in mind that one square meter = 10.76391 square feet, you can write a function that starts with the area in square meters, and gives as output the area in square feet. The code looks like this:

The code above defines a function called m2toft2 that takes in a single input, called area_meters, and returns a single output, called area_feet. Let's try another one:

A function by itself can be difficult to understand, so let's add some comments describing what the function does.

This way, if you forget what m2toft2 does, Python will be able to remind you, like this:

This can be especially useful for large programs with lots of functions, some of which might have multiple arguments. For example, a function might take a list of areas of properties in square meters and a list of prices per square meter, and return lists with area in square feet and price per square foot. That function would look like this:

Let's try it and see what happens:

Practice

Python comes with many predefined functions. Try this one:

Now write a function that returns the greatest per-unit-area property price for a list of property prices per-unit-area, and then use your function for the list price_usd_per_m2.

Practice

The previous example does not extend the max function that is in Python. Keeping in mind that list comprehensions can be used to iterate through lists, use list comprehension or loops to write a function which, given a list of property areas and a corresponding list of property prices per unit area, returns the total price of the most expensive property.

Lambda Functions ¶

The function definitions we've been working with so far are fine for most purposes, but they can easily become a little long. When that happens, you might want to use a shorter method to expressing a function; that's what lambda functions are for. Here's code for a function which adds 3 to a number.

Now that we've defined our function, let's try it out. If we wanted the function to add 3 to 5, the code would look like this:

Practice

Try it yourself! Write a lambda function called sub_4 which will subtract 4 from a given number, and then try it out with the number 7.

Working with Errors¶

Error Handling¶

Error handling is a very important part of coding. It will make sure our code runs smoothly even with edge cases. try and except are the syntax we use in error handling. Let's create a function to demonstrate how this works.

We start with a function that calculates the quotient of two numbers. There are two inputs of the function: nominator and denominator. The function works only when:

• both inputs are numerical numbers
• the denominator is not zero

We can use try and except to make sure the function runs smoothly even with error inputs.